Layer-Stabilizing Deep Learning
نویسندگان
چکیده
منابع مشابه
Layer-wise learning of deep generative models
When using deep, multi-layered architectures to build generative models of data, it is difficult to train all layers at once. We propose a layer-wise training procedure admitting a performance guarantee compared to the global optimum. It is based on an optimistic proxy of future performance, the best latent marginal. We interpret autoencoders in this setting as generative models, by showing tha...
متن کاملDeep Learning Layer-wise Learning of Feature Hierarchies
Hierarchical neural networks for object recognition have a long history. In recent years, novel methods for incrementally learning a hierarchy of features from unlabeled inputs were proposed as good starting point for supervised training. These deep learning methods— together with the advances of parallel computers—made it possible to successfully attack problems that were not practical before,...
متن کاملImportance of Cross-Layer Cooperation for Learning Deep Feature Hierarchies
A common property of hierarchical models of the brain is their capacity to integrate bottom-up and top-down information in order to distill the task-relevant information from the sensory noise. In this paper, we argue that such cooperation between upper and lower layers is not only useful at prediction time but also at learning time in order to build a successful feature hierarchy. The claim is...
متن کاملBeyond Shared Hierarchies: Deep Multitask Learning through Soft Layer Ordering
Existing deep multitask learning (MTL) approaches align layers shared between tasks in a parallel ordering. Such an organization significantly constricts the types of shared structure that can be learned. The necessity of parallel ordering for deep MTL is first tested by comparing it with permuted ordering of shared layers. The results indicate that a flexible ordering can enable more effective...
متن کاملDeep Trans-layer Unsupervised Networks for Representation Learning
Learning features from massive unlabelled data is a vast prevalent topic for highlevel tasks in many machine learning applications. The recent great improvements on benchmark data sets achieved by increasingly complex unsupervised learning methods and deep learning models with lots of parameters usually requires many tedious tricks and much expertise to tune. However, filters learned by these c...
متن کاملذخیره در منابع من
با ذخیره ی این منبع در منابع من، دسترسی به آن را برای استفاده های بعدی آسان تر کنید
ژورنال
عنوان ژورنال: IFAC-PapersOnLine
سال: 2019
ISSN: 2405-8963
DOI: 10.1016/j.ifacol.2019.12.664